[securityAssistant] LangGraph ES|QL query generation tool #186489

patrykkopycinski · 2024-06-19T21:02:00Z

Important

Upping cloud deploy ML memory for ELSER and enabling eval FF. Must revert 45e586b before merging!

Summary

Adds LangGraph-based ESQL generation tool that was heavily inspired by Observability AI Assistant query function

Currently functionality is hidden behind FF:
xpack.securitySolution.enableExperimental=['aiAssistantGraphEsqlTool']

This tool relies also on Knowledge base to be enabled.

To test Cases changes feel free to just post a comment like:

```esql
FROM logs-*
| STATS count = COUNT(*) BY host.name
| SORT count DESC
| LIMIT 10

(make sure to close esql tag, because github formatting didn't allow me to do so)

esql.generation.1.mp4

… to use kbDataClient instead of esStore

…ggraph # Conflicts: # yarn.lock

…raph # Conflicts: # x-pack/plugins/observability_solution/observability_ai_assistant_app/public/functions/visualize_esql.tsx

…raph

…raph # Conflicts: # package.json # x-pack/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/graph.ts # x-pack/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/index.ts # x-pack/plugins/search_playground/server/lib/conversational_chain.test.ts # x-pack/test/security_solution_cypress/config.ts # yarn.lock

…raph

patrykkopycinski · 2024-06-24T09:07:22Z

/ci

patrykkopycinski · 2024-06-24T11:44:06Z

/ci

kibana-ci · 2024-07-01T23:49:47Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 53b998a

Failed CI Steps

Defend Workflows Cypress Tests #19

Test Failures

[job] [logs] Defend Workflows Cypress Tests #19 / When accessing Endpoint Response Console from Cases "before all" hook for "should display responder option in take action menu" "before all" hook for "should display responder option in take action menu"

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	5578	5579	+1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/elastic-assistant`	139	141	+2
`elasticAssistant`	34	39	+5
total			+7

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`cases`	479.4KB	479.8KB	+400.0B
`securitySolution`	15.5MB	15.6MB	+20.8KB
total			+21.2KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`elasticAssistant`	1	2	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	83.6KB	83.8KB	+171.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/elastic-assistant`	166	168	+2
`elasticAssistant`	48	53	+5
total			+7

async chunk count

id	before	after	diff
`securitySolution`	101	102	+1

ESLint disabled in files

id	before	after	diff
`elasticAssistant`	0	1	+1

ESLint disabled line counts

id	before	after	diff
`securitySolution`	551	554	+3

Total ESLint disabled count

id	before	after	diff
`elasticAssistant`	49	50	+1
`securitySolution`	634	637	+3
total			+4

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @patrykkopycinski

peluja1012

We plan to discuss moving common esql validation code before merging.

…raph # Conflicts: # x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts # x-pack/plugins/elastic_assistant/server/lib/langchain/executors/types.ts # x-pack/plugins/elastic_assistant/server/lib/langchain/graphs/default_assistant_graph/index.ts # x-pack/plugins/elastic_assistant/server/routes/post_actions_connector_execute.test.ts # x-pack/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts

joemcelroy

Reviewed Playground changes and working fine. the test updates look fine too.

angorayc

Thanks for the enhancement @patrykkopycinski 👍 👍 👍
I enabled the knowledge base and preview the case, but still had an error. Could you please check if you can reproduce this?

Screen.Recording.2024-07-04.at.17.22.49.mov

angorayc · 2024-07-04T16:28:28Z

...plugins/security_solution/public/assistant/get_comments/custom_codeblock/esql_code_block.tsx

+            </EuiCodeBlock>
+          </EuiFlexItem>
+
+          {!showVisualization && (


If it relies on knowledge base to work, should we check if the knowledge base toggle is on here?

Thank you @angorayc, the UI part doesn't need Knowledge base to be enabled, but it requires in your case to packetbeat-* to exists and have proper data propagated

Yes, I can confirm that I have packetbeat-* data. I searched the same ESQL within timeline, and there was one entry appeared in the table, but it wasn't able to be visualised from the case markdown when using preview.

will double check, Thank you 🙇

peluja1012 · 2024-07-08T22:39:50Z

/ci

patrykkopycinski · 2024-07-11T16:20:08Z

/ci

…aseByDefault and assistantModelEvaluation FF

spong · 2024-07-11T20:45:54Z

/ci

elasticmachine · 2024-07-11T21:36:16Z

⏳ Build in-progress, with failures

Failed CI Steps

History

💚 Build #220992 succeeded 45e586b
💔 Build #220954 failed 9bcdf68
💔 Build #220235 failed 9bcdf68

cc @patrykkopycinski

delanni · 2024-09-05T08:33:21Z

.buildkite/scripts/steps/cloud/deploy.json

@@ -141,7 +141,7 @@
              ],
              "id": "ml",
              "size": {
-                "value": 1024,


afaik, this is no longer needed, with the autoscaling we have in place

spong and others added 23 commits May 30, 2024 17:23

Migrates to LangSmith and adds KB Tools

0b03e98

Merge branch 'main' of github.com:elastic/kibana into kb-tools

b888052

yarn.lock update

de9d530

Port title generation into langgraph node

b862a29

Merge branch 'main' of github.com:elastic/kibana into kb-tools

0b0bdab

yarn.lock update

6451142

Primes initial context with required kb docs and updates kb retrieval…

14b0c65

… to use kbDataClient instead of esStore

Merge branch 'main' of github.com:elastic/kibana into kb-tools

62754d3

Remove chat title generation node

83cdc37

yarn.lock update

b04da0e

Merge branch 'main' into kb-tools

c564e2e

yarn.lock update2

5bc866b

fix types

8477e13

Bump

9875273

Merge branch 'kb-tools' of github.com:spong/kibana into feat/esql-lan…

bfee899

…ggraph # Conflicts: # yarn.lock

WIP

5ba88d8

Merge branch 'main' of github.com:elastic/kibana into feat/esql-langg…

80698c0

…raph # Conflicts: # x-pack/plugins/observability_solution/observability_ai_assistant_app/public/functions/visualize_esql.tsx

Merge branch 'main' of github.com:elastic/kibana into feat/esql-langg…

ae32a5d

…raph

WIP

aa97cc0

Merge branch 'main' of github.com:elastic/kibana into feat/esql-langg…

22d4952

…raph

add cases

825afe3

cleanup

6468286

patrykkopycinski added the ci:cloud-deploy Create or update a Cloud deployment label Jun 20, 2024

patrykkopycinski added 2 commits June 22, 2024 22:09

Merge branch 'main' of github.com:elastic/kibana into feat/esql-langg…

fc6404d

…raph

WIP

5421b42

cleanup

2e88bfe

fix

48259c9

patrykkopycinski changed the title ~~Feat/esql langgraph~~ [securityAssistant] LangGraph ES|QL query generation tool Jul 2, 2024

patrykkopycinski self-assigned this Jul 2, 2024

patrykkopycinski added release_note:enhancement Team:Security Generative AI Security Generative AI v8.15.0 labels Jul 2, 2024

patrykkopycinski marked this pull request as ready for review July 2, 2024 00:03

patrykkopycinski requested review from a team as code owners July 2, 2024 00:03

peluja1012 requested changes Jul 2, 2024

View reviewed changes

pgayvallet approved these changes Jul 2, 2024

View reviewed changes

joemcelroy approved these changes Jul 2, 2024

View reviewed changes

angorayc reviewed Jul 4, 2024

View reviewed changes

peluja1012 added the ci:cloud-deploy Create or update a Cloud deployment label Jul 8, 2024

Up ML node memory for ci-cloud-deploy, and enable assistantKnowledgeB…

45e586b

…aseByDefault and assistantModelEvaluation FF

spong requested a review from a team as a code owner July 11, 2024 19:43

spong added ci:cloud-redeploy Always create a new Cloud deployment and removed ci:cloud-deploy Create or update a Cloud deployment labels Jul 11, 2024

delanni reviewed Sep 5, 2024

View reviewed changes

patrykkopycinski closed this Nov 22, 2024

patrykkopycinski deleted the feat/esql-langgraph branch November 22, 2024 15:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[securityAssistant] LangGraph ES|QL query generation tool #186489

[securityAssistant] LangGraph ES|QL query generation tool #186489

patrykkopycinski commented Jun 19, 2024 •

edited by spong

Loading

patrykkopycinski commented Jun 24, 2024

patrykkopycinski commented Jun 24, 2024

kibana-ci commented Jul 1, 2024 •

edited

Loading

API count

async chunk count

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

peluja1012 left a comment

joemcelroy left a comment

angorayc left a comment

angorayc Jul 4, 2024

patrykkopycinski Jul 9, 2024

angorayc Jul 9, 2024

patrykkopycinski Jul 9, 2024

peluja1012 commented Jul 8, 2024

patrykkopycinski commented Jul 11, 2024

spong commented Jul 11, 2024

elasticmachine commented Jul 11, 2024 •

edited

Loading

delanni Sep 5, 2024

[securityAssistant] LangGraph ES|QL query generation tool #186489

[securityAssistant] LangGraph ES|QL query generation tool #186489

Conversation

patrykkopycinski commented Jun 19, 2024 • edited by spong Loading

Summary

patrykkopycinski commented Jun 24, 2024

patrykkopycinski commented Jun 24, 2024

kibana-ci commented Jul 1, 2024 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

async chunk count

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

History

peluja1012 left a comment

Choose a reason for hiding this comment

joemcelroy left a comment

Choose a reason for hiding this comment

angorayc left a comment

Choose a reason for hiding this comment

angorayc Jul 4, 2024

Choose a reason for hiding this comment

patrykkopycinski Jul 9, 2024

Choose a reason for hiding this comment

angorayc Jul 9, 2024

Choose a reason for hiding this comment

patrykkopycinski Jul 9, 2024

Choose a reason for hiding this comment

peluja1012 commented Jul 8, 2024

patrykkopycinski commented Jul 11, 2024

spong commented Jul 11, 2024

elasticmachine commented Jul 11, 2024 • edited Loading

⏳ Build in-progress, with failures

Failed CI Steps

History

delanni Sep 5, 2024

Choose a reason for hiding this comment

patrykkopycinski commented Jun 19, 2024 •

edited by spong

Loading

kibana-ci commented Jul 1, 2024 •

edited

Loading

elasticmachine commented Jul 11, 2024 •

edited

Loading